Graphic Search Initiation

ABSTRACT

A system for retrieving information. The system comprises a storage unit for storing an image that includes first and second selectable objects. The first selectable object is associated with a first metadata and the second selectable object is associated with a second metadata. The system also includes a visual output device and a processor that communicates with the visual output device and the storage unit to read the first and second metadata. The system also includes a user input device that communicates with the processor and instructs the processor to read the first and second metadata and display the first and second metadata on the visual output device.

BACKGROUND

A common problem that users encounter when searching for informationresources is how to choose keywords for input to a search engine. Oneparticularly perplexing problem occurs when a user wants to search foritems for which the user has seen but has no text-based information fromwhich to formulate a search. For example, while browsing the web, a usermay see a jacket for which the user wants more information. Unless thejacket is the subject of an advertisement, the user will have greatdifficulty formulating a query without knowledge of, for example, themanufacturer. Even if the user could identify the manufacturer,searching for additional information about a particular product amongsimilar products offered by that manufacturer can be time-consuming andfrustrating. Accordingly, a system and method is needed to solve thisproblem.

SUMMARY

One embodiment of the present invention is directed to a system forretrieving information. The system comprises a storage unit for storingan image that includes first and second selectable objects. The firstselectable object is associated with a first metadata and the secondselectable object is associated with a second metadata. The system alsoincludes a visual output device and a processor that communicates withthe visual output device and the storage unit to read the first andsecond metadata. The system also includes a user input device thatcommunicates with the processor and instructs the processor to read thefirst and second metadata and display the first and second metadata onthe visual output device.

DESCRIPTION OF THE FIGURES

For the present invention to be understood clearly and readilypracticed, the present invention will be described in conjunction withthe following figures, wherein:

FIG. 1 is a functional block diagram of a data processing system forhosting web pages according to one embodiment of the present invention;

FIG. 2 is a functional block diagram of various hardware components ofan information processing system used in accordance with one embodimentof the present invention;

FIG. 3 is a combination flow chart and logical block diagram thatillustrates the formation of a metadata-enabled image according to oneembodiment of the present invention;

FIG. 4 illustrates a web page that includes a metadata-enabled imagedisplayed thereon in accordance with one embodiment of the presentinvention;

FIG. 5 illustrates a web page that includes discrete selectable objectsdisplayed thereon according to one embodiment of the present invention;and

FIG. 6 is an exemplary flow diagram that illustrates a process in whicha user utilizes the present invention to access rich content about animage.

DETAILED DESCRIPTION

It is to be understood that the figures and descriptions of the presentinvention have been simplified to illustrate elements that are relevantfor a clear understanding of the present invention while eliminating,for purposes of clarity, other elements. For example, certain detailsrelating to the operation and design of a network capable of carryinghypertext traffic, such as the Internet, the specifications of hypertextprotocols, such as HTTP, for use in transporting HTML pages, and theconstruction of a browser with plug-in extensibility, such as InternetExplorer, are not described herein. Those of ordinary skill in the artwill recognize, however, that these and other elements may be desirablein an interactive networked environment. A discussion of such elementsis not provided because such elements are well known in the art andbecause they do not facilitate a better understanding of the presentinvention.

One embodiment of the present invention relates to a system and methodthat allow a user of a web browser to select a digitized still image ormotion video, or a portion thereof, and automatically formulate a searchquery for information resources associated with the selected image,video or the selected portion. Accordingly, the term “image,” as usedherein, refers to either a still image or a video frame within a streamof video frames. Specifically, the systems and methods described hereinrelate to images with an enhanced data set referred to as “metadata.”For a general understanding of the features of the present invention,reference is made to the drawings, wherein like reference numerals havebeen used throughout to identify similar elements.

FIG. 1 illustrates a networked system architecture 100 in which thepresent invention operates according to one embodiment of the presentinvention. System 100 includes a client computer 102 connected to aremote server computer 106 over a computer network 108. Client 102includes an HTTP browser application program 104 operating thereon,which may be any application program that allows for multimediapresentation of information, including text, images, sound, and videoclips such as Netscape Navigator, Microsoft Internet Explorer or anequivalent. System 100 also includes a server 110, with a metadataediting application program 112 operating thereon, connected to remoteserver 106 over network 108. An image database server 114, alsoconnected to network 108, stores images or other multimedia files 116.Metadata editor 112 is an application program designed to attach orotherwise associate metadata to the images 116. Image database 114 is aany body of information that is organized so that it can be retrieved,stored and searched in a coherent manner by a “database engine”—i.e. acollection of software methods for retrieving or manipulating data inthe database. For example, image database server 114 may be arelational, object-oriented, or object-relational database.

It is understood that computer network 100 illustrated in FIG. 1 isexemplary, and alternative configurations may also be used in accordancewith the invention. For example, network 108, as those skilled in theart will understand, may be any suitable computer network including, forexample, a metropolitan area network, and/or various “Internet” or IPnetworks such as the World Wide Web, a private Internet, a secureInternet, a value-added network, a virtual private network, an extranet,or an intranet. Other suitable networks may contain other combinationsof servers, clients, and/or peer-to-peer nodes. The present inventionmay also be practiced in distributed computing environments where tasksare performed by remote processing devices that are linked through acommunications network. In a distributed computing environment, programmodules may be located in both local and remote memory storage devices.

Network 108 may include communications or networking software such asthe software available from Novell, Microsoft, Artisoft, and othervendors. A larger network, such as a wide area network (WAN), maycombine smaller network(s) and/or devices such as routers and bridges,large or small, and the networks may operate using TCP/IP, SPX, IPX, andother protocols over twisted pair, coaxial, or optical fiber cables,telephone lines, satellites, microwave relays, modulated AC power lines,physical media transfer, and/or other data carrying transmission “wires”known to those of skill in the art. For convenience “wires” includesinfrared, radio frequency, and other wireless links or connections.

FIG. 2 is a block diagram that illustrates various hardware componentsof an information processing system 200 used in accordance with thepresent invention. Information processing system 200 is representativehardware of client 102, client 110, or server 106. System 200 may be anynetwork-enabled device such as, for example, a personal computer, aprogrammable digital assistant (PDA), a mainframe, a workstation, alaptop computer, a hand-held computing device, or combinations thereof.System 200 can optionally include, for example, a processing unit 204, amonitor 206, and a user interface 208. Processing unit 204 includes aprocessor 210 in communication with a memory 212 (shown in phantom)that, in turn, includes a volatile memory 214 and a storage unit 216.These are representative components of a computer whose operation iswell understood.

Processor 210 may include a general purpose device such as an IntelPentium® processor or other “off-the-shelf” microprocessor. Processor210 may include a special purpose processing device such as, forexample, an ASIC, PAL, PLA, PLD or other customized or programmabledevice. Memory 212 may include, for example, a static RAM, a dynamicRAM, a flash memory, a ROM, a CD-ROM, a disk, a tape, a magnetic,optical, or another computer storage medium. User interface 208 mayinclude, for example, a keyboard, a mouse, a touch screen, a light pen,a tablet, a microphone, a position sensor, a pressure sensor, a thermalsensor, or other input hardware with accompanying firmware and/orsoftware. Monitor 206 or other type of display device is connected toprocessor 210 via an interface, such as a video adapter.

System 200 may also include a computer readable medium having executableinstructions or data fields stored thereon, such as storage unit 216.The computer readable medium can be any available medium that can beaccessed by a general purpose or special purpose computer. By way ofexample, and not limitation, such a computer readable medium cancomprise RAM, ROM, electrically erasable programmable read only memory(EEPROM), CD-ROM or other optical disk storage, magnetic disk storage orother magnetic storage devices, or any other medium that can be used tostore desired executable instructions or data fields and that can beaccessed by a general purpose or a special purpose computer.

The computer readable medium tangibly embodies a program, functions,and/or instructions that cause the computer system to operate in aspecific and predefined manner as described herein. Those skilled in theart will appreciate, however, that the process described below may beimplemented at any level, ranging from hardware to application softwareand in any appropriate physical location. For example, the presentinvention may be implemented as software code to be executed by system200 using any suitable computer language such as, for example,microcode, and may be stored on any of the storage media describedabove, or can be configured into the logic of system 200. According toanother embodiment, the instructions may be implemented as software codeto be executed by system 200 using any suitable computer language suchas, for example, Java, Pascal, C++, C, Perl, database languages, APIs,various system-level SDKs, assembly, firmware, microcode, and/or otherlanguages and tools.

FIG. 2 and the foregoing discussion are intended to provide a brief,general description of a suitable computing environment in which theinvention may be implemented. Although not required, the invention willbe described in the general context of computer-executable instructions,such as program modules, being executed by a personal computer.Generally, program modules include routines, programs, objects,components, data structures, etc. that perform particular tasks orimplement particular abstract data types. Additionally, those skilled inthe art will appreciate that the present invention is not limited to aparticular computer system platform, processor, operating system, ornetwork.

FIG. 3 illustrates a process 300 in which metadata editor 112 combines aset of metadata 302 with an image 304 to produce a metadata-enabledimage 306 according to an embodiment of the present invention. Metadataeditor 112 combines metadata 302 with image 304 using, for example, theResource Description Framework (RDF) and extensible Markup Language(XML) in a manner known to those skilled in the art, RDF is aninteroperable standard for metadata on the web defined by the World WideWeb Consortium (W3C). Document Type Definitions (DTDs) may also be usedwith XML to create a suitable metadata model. Using these standards, anyweb image, ranging from gas grills to footwear, can be associated withmetadata to conveniently display rich content about the image.

FIG. 4 illustrates a web page 400 that includes metadata-enabled image306 displayed thereon and in accordance with one embodiment of thepresent invention. Web page 400 may also display a cursor 402 that maybe controlled by user input, as described above. The user may utilizecursor 402 to select an image within web page 400, such asmetadata-enabled image 306. According to one embodiment, the userpositions cursor 402 over metadata-enabled image 306 and clicks on theimage 306 using a right-hand button on a pointing device to indicate,for example, a request for context-sensitive action.

According to one embodiment, the context-sensitive action produces auser interface control that may include, for example, a pop-up windowhaving a list of options related to image 306 and its associatedmetadata 302. Those of skill in the art will realize that such userinterface selection controls may be implemented using, for example, anActiveX control, a Java control, an applet, or a browser plug-in, or aseparate software application. The list of options related to metadata302 may include, for example, launching a suitable metadata extractiontool to view metadata 302.

According to another embodiment, the list of options associated with thecontext sensitive action may include automatically inserting themetadata into a search engine. According to such an embodiment, a searchis automatically initiated to locate resources, such as web sites,within a distributed environment. For example, the user may designate asearch engine as a user preference before initiating thecontext-sensitive action. Example of such search engines includeGoogle™, offered by Google of Mountain View, Calif., which may beaccessed at the URL http://www.google.com and RealPages, offered byBellSouth, which may be accessed at http://www.realpages.com. Once theuser designates the search engine, the present invention may be used toautomatically insert metadata 302 into the search engine as searchterms. According to one embodiment, the present invention may launch anew browser window that displays the search engine with metadata 302entered as search terms and allows the user to edit the terms beforeinitiating the search. According to another embodiment, the search isinitiated immediately from the context-sensitive action. The searchengine examines the search criteria and returns a list of web documentsto browser 104 at the client computer 102 that conform to the searchcriteria and that may be desired by the user.

FIG. 5 illustrates a web page 500 that includes an exemplarymetadata-enabled image 502 displayed thereon and in accordance with anembodiment of the present invention. Image 502 includes multiplediscrete selectable objects 504 and 506. Each selectable object has aunique set of metadata associated with it. Accordingly, the user canaccess metadata associated with any selectable object in the scene. Itshould be understood that web pages 400 and 500 are merely exemplary ofthe displays and methods that may be used to select and access metadataassociated with an image. Thus, any suitable display and method ofdisplaying images may be used in accordance with the present invention.

FIG. 6 is an exemplary flow diagram that illustrates a process 600 inwhich a user uses the present invention to access rich content about animage. The process begins at step 602 in which a user at client 102directs browser 104 to display, for example, web page 500 that includesan image 502. As explained above, metadata is attached to and/orassociated with selectable objects 504 and 506 within the image 502. Toview metadata about object 504 (a woman's jacket), in step 604, the userclicks on object 504 using, for example, a right-hand button on thepointing device and selects among a list of user options. In step 606,according to one embodiment, the present invention inserts the metadataassociated with selectable object 504 into a predetermined searchengine. Finally, in step 608, according to another embodiment, thepresent invention automatically initiates the search using metadata asinput.

It should be understood that the present invention is not limited by theforegoing description, but embraces all such alterations, modifications,and variations in accordance with the spirit and scope of the appendedclaims.

1-14. (canceled)
 15. An electronic computer system for retrievinginformation, comprising: means for storing at least one image, whereinan image comprises at least one selectable object having associatedmetadata according to a metadata model formed with extensible MarkupLanguage and Document Type Definitions; a visual output device;electronic means, in communication with the visual output device and thestoring means, for processing information, wherein the processing meansis configured to read the metadata; user input means, in communicationwith the processing means, for instructing the processing means to readthe metadata and to display the metadata on the visual output device;wherein the processing means initiates an information search based onthe metadata upon reading the metadata.
 16. The system of claim 15,wherein the processing means comprises a metadata editor configured toassociate the metadata with the selectable object.
 17. The system ofclaim 15, wherein the metadata conforms to a schema expressed inaccordance with a Resource Description Framework and SyntaxSpecification.
 18. A method in an electronic computer system forretrieving information, comprising: selecting a selectable image objectdisplayed on a viewing device, the selectable image object havingassociated metadata according to a metadata model formed with extensibleMarkup Language and Document Type Definitions; reading the metadata;initiating an information search based on the read metadata upon readingthe metadata; and displaying, on the viewing device, information inconnection with the information search.
 19. The method of claim 18,wherein the selecting step comprises selecting at least one selectableimage object from among a plurality of selectable image objectsdisplayed on the viewing device, and each of the plurality of selectableimage objects has respective associated metadata according to a metadatamodel formed with extensible Markup Language and Document TypeDefinitions.
 20. The method of claim 18, wherein the displaying stepcomprises displaying the metadata associated with the selectedselectable image object.
 21. The method of claim 18, wherein themetadata conforms to a schema expressed in accordance with a ResourceDescription Framework and Syntax Specification.
 22. A computer-readablemedium having stored thereon instructions which, when executed by anelectronic processor, cause the processor to perform: selecting aselectable image object displayed on a viewing device, the selectableimage object having associated metadata according to a metadata modelformed with extensible Markup Language and Document Type Definitions;reading the metadata; initiating an information search based on the readmetadata upon reading the metadata; and displaying, on the viewingdevice, information in connection with the information search.
 23. Themedium of claim 22, wherein the selecting step comprises selecting atleast one selectable image object from among a plurality of selectableimage objects displayed on the viewing device, and each of the pluralityof selectable image objects has respective associated metadata accordingto a metadata model formed with extensible Markup Language and DocumentType Definitions.
 24. The medium of claim 22, wherein the displayingstep comprises displaying the metadata associated with the selectedselectable image object.
 25. The medium of claim 22, wherein themetadata conforms to a schema expressed in accordance with a ResourceDescription Framework and Syntax Specification.